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Abstract — A decode and forward protocol based Trellis Coded 
Modulation (TCM) scheme for the half -duplex relay channel, in a 
Rayleigh fading environment, is presented. The proposed scheme 
can achieve any spectral efficiency greater than or equal to one 
bit per channel use (bpcu). A near-ML decoder for the suggested 
TCM scheme is proposed. It is shown that the high SNR per- 
formance of this near-ML decoder approaches the performance 
of the optimal ML decoder. The high SNR performance of this 
near-ML decoder is independent of the strength of the Source- 
Relay link and approaches the performance of the optimal ML 
decoder with an ideal Source-Relay link. Based on the derived 
Pair-wise Error Probability (PEP) bounds, design criteria to 
maximize the diversity and coding gains are obtained. Simulation 
results show a large gain in SNR for the proposed TCM scheme 
over uncoded communication as well as the direct transmission 
without the relay. Also, it is shown that even for the uncoded 
transmission scheme, the choice of the labelling scheme (mapping 
from bits to complex symbols) used at the source and the relay 
significantly impacts the BER vs SNR performance. We provide 
a good labelling scheme for 2 ; -PSK signal set, where I > 2 is an 
integer. 

I. PRELIMINARIES AND BACKGROUND 




Fig. 1. The Relay Channel 



We consider the Rayleigh fading relay channel shown in 
Fig. Q] consisting of the source node S, the relay node R and 
the destination node D. It is assumed that R can operate only 
in the half-duplex mode, i.e., it cannot receive and transmit 
simultaneously in the same frequency band. It is assumed 
that R has perfect knowledge about the instantaneous value 
of the fade coefficient associated with the S-R link and D 
has perfect knowledge about the instantaneous values of the 
fade coefficients associated with the S-R, R-D and S-D links. 
Throughout, the phase during which the relay is in reception 
mode is referred to as Phase 1 and the phase during which the 
relay is in transmission mode is referred to as Phase 2. 

In all practical scenarios the S-R and R-D links are stronger 
than the S-D link. As a result, the use of coding schemes 
which involve both S and R, can potentially outperform the 



coding schemes involving S alone. The problem addressed 
in this paper is the design of a Trellis Coded Modulation 
(TCM) scheme for the relay channel, which achieves a spectral 
efficiency greater than or equal to one bit per channel use 
(bpcu) and provides a large gain compared to the TCM scheme 
for the direct transmission without the relay. 

A comparison of the proposed TCM scheme is made 
with the uncoded transmission scheme for the relay channel 
operating in the Decode and Forward (DF) mode, in which no 
coding is done at S and R, which is described as follows: At S, 
uncoded bits are directly mapped onto complex symbols from 
a signal set during Phase 1 and transmitted. During Phase 2, 
S and R map the uncoded bits (S uses the same bits as in 
Phase 1 and R uses the bits it decoded during Phase 1) onto 
complex symbols from a signal set for transmission. The signal 
sets used at S during Phase 1, S during Phase 2 and R during 
Phase 2 are assumed to be the same, but the mapping from 
bits to complex symbols can be different. 

A. The Proposed Scheme 

The proposed TCM scheme for the fading relay channel is 
shown in Fig. |2(a)| and Fig. |2(b)| (shown at the top of the next 
page). It is assumed that TCM encoding at S during Phase 1 
and Phase 2 and at R during Phase 2 take place using the same 
encoder, but the labelling schemes used for mapping coded bits 
onto signal points can be different. By a trellis, we refer to the 
states of the TCM encoder, the edges connecting these states 
in two successive stages and the complex numbers which are 
labelled on these edges. By a path in the trellis, we refer to the 
sequence of connected edges (the complex numbers labelled 
on the edges are not included). Let Ts lt Ts 2 and Tr represent 
the trellises used for encoding by S during Phase 1, by S 
during Phase 2 and by R during Phase 2 respectively. Since 
the encoder corresponding to the trellises Ts ± , 7s 2 ar, d Tr are 
the same, they differ only in the complex numbers labelled on 
the edges. Let £ denote the set of edges connecting the states in 
two successive stages of the trellis. The labelling of the edges 
in the trellis Tsi is given by the map X Sl : ( — > S, where S 
is the signal set used at S and R. Similarly, the maps X S2 and 
X r are defined for the the trellises Ts 2 and Tr respectively. 

During Phase 1, input bits at S are encoded by the TCM 
encoder, mapped onto complex symbols from the signal set 
S and interleaved by the block interleaver as shown in Fig. 
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Fig. 2. The TCM Scheme 



|2(a)| Each transmission made is assumed to be a block of L 
complex symbols. After each transmission, the encoder state is 
brought back to the all zeros state by appending zero input bits 
at the end. Let Vs denote the path in the trellis corresponding 
to the output of the TCM encoder at S. Let x\ (Vs) denote 
the symbol to be transmitted corresponding to the i th branch 
of the path Vs, where 1 < i < L. 

The received complex signal sequence at R and D during 
Phase 1 are given by, 



Y r l = hi r xi x (V S ) + Zr, 



1 < i < L 



where h\ r and h % sdl are the independent zero mean circularly 
symmetric complex Gaussian fading coefficients associated 
with the S-R and S-D links respectively with the corresponding 
variances given by a 2 T and a 2 d . The additive noises z r and 
Zd ± are assumed to be CN(0, 1), where CN(0, 1) denotes 
the standard circularly symmetric complex Gaussian random 
variable. An interleaver with sufficiently large block length is 
assumed to make the block fading channel appear as a fading 
channel with independent fade coefficients for successive 
channel uses. At R, after deinterleaving, an ML decoder using 
the Viterbi algorithm is used to decode the bits (Fig. |2(a)| >. 
Let Vr denote the path in the trellis to which R decodes. At 
D, the received complex numbers are deinterleaved and stored 
(Fig.pJ. 

During Phase 2, S encodes the same bits used during 
Phase 1 and R encodes the bits it decoded during Phase 1, 
using the TCM encoder. The encoded bits at S and R are 
mapped onto complex symbols from the signal set S (possibly 
with different mappings and also possibly different from the 
mapping used at S during Phase 1) and interleaved using the 



block interleavers (same as the interleaver used at S during 
Phase 1) as shown in Fig. |2(b)| Let x\ (Vs) and x\,(Vr) 
denote the symbols transmitted by S and R respectively during 
Phase 2, corresponding to the i th branch of the paths Vs and 
Vr, where 1 < i < L, 

The received complex signal sequence at D during Phase 2 
is given by, 



sd2 and Kd 



Yd 2 = hl d2 xl 2 (V s ) + K d xl(V R ) + z d2 , 

where 1 < i < L. The random variables h 
are the independent zero mean circularly symmetric complex 
Gaussian fading coefficients associated with the S-D and R-D 
links respectively with the corresponding variances given by 
a 2 d and a 2 d . The additive noise Zd 2 is CN(Q, 1). Throughout, 
it is assumed that a 2 d <C a 2 d , due to the proximity of the relay 
to the destination than the source to the destination. 

During Phase 2, at D the received complex numbers are 
deinterleaved and along with the complex numbers stored 
during Phase 1 are fed to the proposed near-ML TCM decoder 
described in Section III. 

B. Background and Related Work 

Achievable rates and upper bounds on the capacity for the 
discrete memoryless relay channel for the full-duplex and 
the more practical half-duplex relay channel were obtained 
in Q~), El- These results were extended for the half duplex 
Gaussian relay channel in 0. Optimal power allocation 
strategies to maximize the achievable rate for the Rayleigh 
fading relay channel were investigated in |4|. Several protocols 
like Amplify and Forward (AF), Decode and Forward (DF), 
Compress and Forward (CF) were proposed Q. Power allo- 
cation strategies for the Non Orthogonal DF (NODF) protocol 



were discussed in J6]. Non-orthogonal relay protocols offer 
higher spectral efficiency when compared with orthogonal 
relay protocols 0, (8], J5J. Hence, the proposed TCM scheme 
in this paper is based on the NODF protocol. 

A turbo-coding scheme for the full duplex and more prac- 
tical half-duplex relay channels were proposed in iPTOll . ifTTI . 
These coding schemes achieve a spectral efficiency strictly less 
than one bpcu, i.e., they pay in bandwidth. Coding schemes for 
the relay channel with bandwidth constraints, which achieve 
a spectral efficiency greater than or equal to one bpcu are not 
reported in the literature, to the best of our knowledge. In this 
paper, we propose a Trellis Coded Modulation (TCM) scheme 
for the half-duplex relay channel for any spectral efficiency 
greater than or equal to one bpcu. 

Implementation as well as the performance analysis of the 
optimal ML decoder for the relay channel operating in the 
DF mode is complicated lfl2l . |fl3l . To solve these problems, 
several sub-optimal decoders for uncoded DF scheme were 
proposed in fl2l . fl3l . Ifl4l and [15|. These problems carry 
over to coded communication using DF protocol as well. If 
the decoder at D is designed assuming that R always decodes 
the bits correctly, the diversity order obtained is dependent on 
whether the relay decoded correctly or not. To circumvent this 
problem, relay selection strategies fl6l . fountain codes IfTTI . 
Cyclic Redundancy Check (CRC) codes lfl8ll etc. are used. To 
solve these problems of finding a decoder with implementable 
complexity and tractable performance analysis, we propose a 
near-ML decoder, whose performance at high SNR approaches 
the performance of the optimal ML decoder. The form of this 
decoder allows a DF scheme in which the relay need not 
check whether it has decoded correctly before forwarding the 
message. This eliminates the need for embedding CRC bits as 
well as the need for feedback from the relay to the source. 
Furthermore, the diversity order of the proposed scheme does 
not depend on whether the relay decoded correctly or not. 

The proposed TCM scheme is different from the Co- 
operative Multiple Trellis Coded Modulation (CMTCM) 
scheme proposed in [18] in the following aspects: 

• The presence of embedded CRC bits and feedback from 
R to S is assumed in [18], whereas we make no such 
assumptions. 

• Interleaving/Deinterleaving is not assumed in ifTSl 
whereas we assume interleaving/deinterleaving which 
makes the quasi-static fading scenario appear as a fast 
fading scenario. 

C. Contributions and Organization 

The main contributions of this paper are as follows: 

• A TCM scheme for the fading relay channel is proposed 
which achieves any spectral efficiency greater than or 
equal to one bpcu. 

• A near-ML decoder is obtained for the proposed TCM 
scheme. The relay need not check whether it has decoded 
correctly before forwarding the message which eliminates 
the need for embedded CRC bits as well as the feedback 
from the relay to the source. 



• The proposed near-ML decoder enables the formulation 
of design criteria to maximize the diversity order and the 
coding gain. 

• Bounds on the PEP are derived for the near-ML decoder 
based on which the criteria to maximize the diversity 
order and coding gain are obtained. 

• The BER vs SNR performance of the near-ML decoder 
proposed with a non-ideal S-R link, at high SNR, ap- 
proaches the performance of the optimal ML decoder 
with an ideal S-R link. This implies that the high SNR 
performance of the near-ML decoder for the relay channel 
with a non-ideal S-R link approaches the performance of 
the optimal ML decoder. 

• It is shown that even for the uncoded transmission 
scheme, a proper choice of the labelling at S and R 
provides a significant performance improvement. We give 
a good labelling scheme for 2 Z -PSK signal set, where 
I > 2 is an integer. 

• Simulation results show a large gain in the BER vs SNR 
performance for the proposed TCM scheme over the 
uncoded transmission scheme as well as the TCM scheme 
for the direct transmission without the relay. 

The organization of the rest of the paper is as follows: In 
Section II, the bounds on the ergodic capacity of the fading 
relay channel with Gaussian input alphabet are compared with 
the capacity bounds when the input is constrained to take 
values from a finite signal set. The description of the near-ML 
decoder for the TCM scheme constitutes Section III. In Section 
IV, the PEP expressions are derived, based on which the code 
design criteria to maximize the diversity order and coding gain 
are obtained. In Section V, effect of the choice of the labelling 
scheme for the uncoded transmission scheme is discussed and 
a good labelling scheme for 2'-PSK constellation is presented. 
In Section VI, TCM code design examples are presented. 
Simulation results are presented in Section VII. 

Notations: For a random variable X s which takes value 
from the set S, we use x St i to represent the z-th element 
of S. E 2 [y] denotes the expectation of Y with respect to 
the random variable z. Throughout, log refers to log-2 and 
C(a) denotes log(l + a). Let CN(0,I n ) denote the stan- 
dard circularly symmetric complex Gaussian random vector 
of length n. Also let N(0, c) denote the scalar real valued 
Gaussian random variable with mean zero and variance c. 
For simplicity, distinction is not made between the random 
variable and a particular realization of the random variable, 
in expressions involving probabilities of random variables. In 
some probability expressions involving conditioning of the 
fading coefficients, the fact that the probability is conditioned 
on the values taken by the fading coefficients is not explicitly 
written, as it can be understood from the context. For a set A, 
\A\ denotes the cardinality of A. For two sets A and B, AUB 
denotes the union of the sets A and B. The superscript (.) T 
denotes the transpose operation. The set {x \ c(x)} denotes 
the set of all values of x for which the condition c(x) is 
satisfied. #t(x) denotes the real part of the complex number 
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x. Throughout, E$ denotes the average energy in dB of the 
signal set S used at the source and the relay. Let Q[.] denote 
the complementary CDF of the standard Gaussian random 
variable. 

II. INFORMATION THEORETIC LIMITS 

Throughout this section, the achievable rate of the decode 
and forward scheme is taken to be the lower bound on the 
capacity and the cut-set bound H) is taken to be the upper 
bound on the capacity. 

For the half duplex relay channel, the received signal at R 
during Phase 1 is given by, 

The received signal at D during Phase 1 and phase 2 are 
given by, 

Ydi = h-sdXsx + z dl ; 

Yd 2 = h ad X 32 + h rd X r + z d2 . 

where X Sl and X S2 denote the symbols transmitted by S 
during Phase 1 and Phase 2 respectively, X r denotes the 
symbol transmitted by R during Phase 2. The random variables 
Zd x , Zd 2 and z r are independent and CN(0, 1). 

For the relay channel with Gaussian input alphabet, the 
lower and upper bounds on the capacity are given by |3| |4|, 

r 1 
Cj = — max min 

2 0</3<l 

{E har [C (\h sr \ 2 E s )} + E hsd [C ((1 - f3)\h sd \ 2 E s )] , 
E hsd [C{\h sd \ 2 E s )] 

+~E htd , hrd [C (\h ad \ 2 E S + \h rd \ 2 E s + 2y/p\h ad \\hrd\Es)] } ; 

(6) 

C§ = ^m^ i mm{E hsdthsr [C((\h sd \ 2 + \h sr \ 2 )E s )] 

+ E hsd [C((l-/3)\h ad \ 2 E s )] , 
Vh sd ,h rd [C (\h sd \ 2 E s + \h rd \ 2 E s + 2y^\h sd \\h rd \E s ^ 

+E hsd [C{\h sd \ 2 E s )]}. 

(7) 



To compute the achievable rate bounds with finite input 
constellation, assume X Sl £ S Sl , X S2 £ S S2 and X r £ S r , 
where <S S1 , S S2 and S r are signal sets such that \S Sl \ = M Sl , 
\S S2 \ = M S2 and |<S r | = M r . Define R 4 = I{X Sl ;Y r ), R 5 = 
I(X S2 ; Y d2 \X r ), R 6 = I(X S1 ; Y dl ), R 7 = I(X S2 , X r ; Y d2 ) and 
Rs = I(X Sl ;Y r ,Yd 1 ). It is assumed that coding schemes in 
which all the possible choices for (X Sl , X S2 , X r ) occur with 
equal probability only are of interest. This forces the marginal 
distributions of X S1 , X S2 , X r to be independent and uniformly 
distributed. 

The achievable rate bounds of the half duplex relay channel 
with finite input constellation can be shown to be fl9l , 

Cf c = min | ±E hsr [Ri] + ±E had [R 2 ] , 

l E h sd m + ^h sd , hrd [iU}}; (8) 

Cg c = min{iE hsr , hsd [R 5 ] + ±E hsd [R 2 ] , 

lv hsd {R3] + ^E hsd , Kd {R 4 ]y (9) 

where R1-R5 are given by ([TJ-((5j, shown at the top of this 
page. 

The expressions for the capacity bounds (O, (0, (0 and (0 
can be evaluated using Monte-Carlo simulations. The plots 
for the Eg Vs Capacity bounds for the relay channel with 
Gaussian alphabet as well as 4-PSK and 8-PSK constellations, 
for a 2 sd =0 dB, (J 2 sr =\5 dB, of d =15 dB, ai~e shown in Fig. [3] 
Fig. [3] also shows the capacity with Gaussian alphabet and 
the constellation constrained capacity with 4-PSK signal set 
for the direct transmission (S-D link). From Fig. [3] it is clear 
that the use of relay can provide significant advantage over the 
direct transmission. From Fig. [3] we can see that for a spectral 
efficiency of 1 bpcu, for a 2 sd =0 dB, <rf r =15 dB, a 2 rd =\5 dB, 
the achievable rate using DF scheme for 4-PSK and 8-PSK 
input constellations are about 3.5 dB and 1.5 dB away from 
that of the Gaussian alphabet. Hence in order to be within 1 .5 



dB away from the achievable rate of the DF scheme using 
Gaussian alphabet, for a spectral efficiency equal to 1 bpcu, 
8 point signal set needs to be used. Hence, in the proposed 
TCM scheme, to achieve a spectral efficiency r bpcu, a signal 
set with 2 2r+1 points is used. 

Note 1: To achieve a spectral efficiency of r bpcu, a total 
of 2r bits get transmitted from S to D in the two Phases of 
relaying. 

The value of Eg required for a BER of 10~ 4 and the 
diversity order for the different schemes achieving a spectral 
efficiency of 1 bpcu are shown in Table [I] at the top of the 
next page (The details regarding the design of TCM schemes 
for the relay channel are presented in Section VI and Eg vs 
BER plots are given in Section VII. The TCM schemes for the 
direct transmission were designed based on the design criteria 
for TCM for fading channels ||2T1 ). From Table [I] it can be 
seen that the proposed schemes for the relay channel, which 
exploit the fact that the S-R and R-D links are better than 
the S-D link, outperform the corresponding schemes for the 
direct transmission without relay. From Fig. [3] we see that the 
capacity of the direct transmission without relay with 4 PSK 
signal set and Gaussian alphabet are respectively 2 dB and 1.5 
dB, which are greater than the value of Eg (0 dB) required 
for the 16 state TCM scheme for the relay channel, for a BER 
of 10 -4 . This means that under the given assumptions, the 
proposed 16 state 8 PSK TCM scheme for the relay channel, 
will outperform even the best possible coding scheme for the 
direct transmission without relay. The claim made above is 
valid even for other values of the variances of the fading links, 
possibly with a higher number of states if not with 16 states. 
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Fig. 3. Rate vs E s for Half Duplex relay Channel with a^ d =0 dB, 0^=15 
dB, ^=15 dB 

III. A NEAR-ML TCM DECODER 

The trellis used for decoding at D is constructed as described 
in the following subsection. The reason why the decoding 
trellis is of the form described will be clear by the end of 
Subsection B of this section. 




Fig. 4. 4-PSK signal set 
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A. Decoding Trellis at D 

Let {a*,a l+1 } and {b l ,b l+1 } respectively denote any two 
pairs of states in the i th and i + I th stages of the trellises 

Tsi and Tr. Let E a \a^ ar *d £fr 6 < + 1 denote the sets of edges 
from a % to a l+1 and b l to b l+1 respectively. 

The trellis used for decoding at D, denoted as Td, 

is constructed as follows: The states of Td at the 



and i + 1 stages are denoted by the two tu- 
ples [a l ,b l ] and [a l+1 ,b l+1 ] respectively. The set of edges 
from [a\6 J ] to [a l+1 , b l+1 ], denoted as £[ a \b i ],[a i + 1 .b i + 1 ], 
contains an edge denoted by the pair {e a » a *+i, e b % b i+i} 
if e i >0 i+i e £ a t <a i+i and e b i >b i+i G £b\b i+1 - The 
edge {e a i Q i+i , efji^i+i } is labelled with the four tuple 
O^si (e a i ia «+i), Af S2 (e i >0 i+i ), ^ (e b > )b »+i ), # r (e b > ib *+i )) e 

Note 2: Even though encoding at S and R during the two 
Phases take place using a total of three trellises, the edges 
connecting the states in two successive stages in the trellis 
Td are constructed based on the trellises 7si and Tr. The 
reason is as follows: The uncoded bits at the input of the TCM 
encoders at S during Phase 1 and Phase 2 are the same. Hence, 
the encoded paths in the trellises 7si and Ts 2 arc always the 
same (though the complex signal sequences may be different, 
due to different labellings) and it is enough to consider one 
of the two trellises Ts 1 and 7s 2 . 

Example 1: For the trellis triple {T^ , 7s 2 , Tr} (shown in 
Fig. 0, labelled with signal points from S (shown in Fig. 0), 
the trellis Td is shown in Fig. [6] 

The decoding metric for the proposed near-ML decoder is 
obtained in the following subsection. 

B. Near-ML Decoding Metric 

By assumption, R has perfect knowledge about the instanta- 
neous value of the fade coefficient associated with the S-R link 
and D has perfect knowledge about the instantaneous values 
of the fade coefficients associated with the S-R, R-D and S-D 
links. Let Al denote the set of all paths of length L in the 
trellis (since the TCM encoder used at S during Phase 1, at 
S during Phase 2 and at R during Phase 2 are the same, the 
set Al is the same for the trellises Ts 1 , Ts 2 an d Tr). Let Vs 
denote the path which corresponds to the symbols transmitted 
by the source during Phase 1 . Let Vr denote the path to which 
the Viterbi decoder at R decodes during Phase 1, i.e., 



Vr = arg min £ \Y? - (V)\ 2 . 

i — I 

The optimal ML decoder at D decides in favour of the path, 

Vs = arg ^ Pr { Y^ ; L , Y^ L \V S } , 

where and Y^ L denote the sequences Y£ , Y% , Y^ 

and Y d \, Yl, Y& respectively and Pr [y^ l , Y^ L \V S ) is 

the probability that Y^ L and Y^ L are the received sequences 
by D during Phase 1 and Phase 2, given that the path 
corresponding to the complex numbers transmitted by S is 
Vs ■ The probability which needs to be maximized is given 

by, 

Pr{Y% L ,Y£ L [P 8 }= J2 Pr{Y^ L ,Y^ L \Vs,-P R }Pr{VH\Vs}. 

(10) 
L 



where Pr (y^ L , Y^ L \V s ,Vr\ is the probability that Y^< 

and Y^ are the received sequences by D during Phase 1 and 
Phase 2, given that the paths corresponding to the complex 
numbers transmitted by S and R are Vs and Vr respectively. 
The quantity Pr {Vr\Vs} is the probability that R decodes 
to the path Vr given that Vs is the path corresponding to the 
complex symbols transmitted by S. 

The probability Pr {Vr\Vs} can be upper bounded by the 
corresponding PEP, i.e., 



Pr{V R \V s }< Y\Q 



Hr « (V S ))\ 



V2 

^ ex P \-\ E Hr « Ws) ~ < (Vr)) 

and we also have, 

p r{Yj; L ,Y^ L \Vs,V R } = 

L i — 1 



(ii) 



Yi 



(12) 



v R eA L K i=i v * 7 J 



: (Ps.Pr) = - h'^x^ {V S )\ 2 + \Y* a - K d2 xl 2 (Vs) - K d < {V R tf + -\h\ r (x^ (Vs) - < (V R )) 



(13) 
(14) 



Substituting ([TT) and (O in ([Toll we get 0, shown at the 
top of the next page. 

If we maximize only the dominant exponential in the upper 
bound (fl~3T >, then the decoded path is given by, 



P s = mg min <J min V" <fo (Vs,Vr) 
V S €A L V R eA L f-f 
1 i=l 



(15) 



where the metric 4>i (Vs,Vr) is given by ( fT4b (shown at the 
top of the next page). 

This near-ML decoder given by ( fT5T l involves minimizing 
the additive metric over two distinct paths and hence the 
decoding takes place in the trellis Td described in the previous 
subsection. Throughout, it is assumed that decoding at D takes 
place using this near-ML decoder. 

The branch metric for the Viterbi decoder corresponding to 
the edge {e a i a «+i, ey. &»+i} in Td can be obtained from (fl4] i 
and is given by 



Lemma 1: Consider a communication system with one 
transmitter and one receiver. The transmitter transmits one 
of the two n-dimensional complex vectors x\ and x 2 E C™ 
corresponding to two messages. The received n-dimensional 
complex vector y = x + z € C™, where x € {xi,X2} and 
z is CN(0, I n ). The decoding rule used at the receiver is 
as follows: The decoder decides in favour of message 1, if 
\\y — x\\\ 2 < ||y — X2|| 2 + c and it decides in favour of message 
2 otherwise, where c e R is a constant. Then the probability 
that the decoder decides in favour of message 2, given that 
message 1 was transmitted is upper bounded as, 



Pr(l 



Proof: 
We have, 



2) < - exp 



\xi - x 2 \ 



\n 2 - K d2 Xs 2 (e a ^ a , + i) - K d X T {e biM+1 )\ 



For example, for the edge from state [1,0] to state [0,1] 
in Td in Example 1, denoted as {ei,o,eo,i}, from Fig. [6] 
the four tuple (X Sl (ei, ), X S2 (ei,o), X Sl (e 0) i), Af r (e ,i)) is 
(s2, S2, S3, S4) and hence the decoding metric is given by, 



P (ei,o, e ,i) 



sdl S 2 



d 2 



^d s 4 



^3)| 2 . 



Pr(l— * 2)= Pr ((I |2/ -si I 



: Pr 
: Pr ^5} 
Pr ( 5R 



V 

2 + 



2 

XI — X2 



2 

X2 — X\ 
\x 2 - Xl| 

> 



c) 1 a; = x\ 


X!) 


N 


X!) 









X = Xl) 



2\\X2 



Since z is CN(0,I n ), it can be shown that 3? ^ z* 

is TV ^0, 1 
Hence, 



X 2 — Xl 
\x 2 - Xl\ 



Let K denote the number of branches which originate from 
each state and let N denote the number of states in the 
trellises Ts 1 , Ts 2 and Tr,. In the N 2 state trellis Td, K 2 
branches originate from a state. As a result, in each step, the 
Viterbi algorithm used for decoding involves N 2 K 2 additions 
and iV 2 comparisons of K 2 values. Hence the decoding 
complexity of the proposed near-ML decoder is approximately 
2N 2 K 2 /(log2K) operations per decoded bit, since log 2 K bits 
get transmitted in the two phases of relaying. 

IV. PEP ANALYSIS OF THE NEAR-ML DECODER 
AND CODE DESIGN CRITERIA 

The following Lemma is useful for the PEP analysis of the 
near-ML decoder. 



Pr(l — ► 2) 



V2 



< exp 

< exp 



\xi - X2\\ c 

2 2||x 2 -xi| 
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2 2||z2-zi| 
2 



\xi - X2\\ _ c 
4 2 



A. PEP Analysis of the Near-ML Decoder 

In this subsection, an upper bound on the PEP that a 
transmitted path Vs at S is decoded by the near-ML decoder 
at D as Vs is derived. 

For the trellis Ts\, let r) si (Vs, Vs) denote the set of values 
of i for which x l sl (Vs) and x l sl (Vs) are different, for 1 < 



i < L. Similarly, r] S2 (Vs,Vs) and rj r (Vs,Vs) are defined for 
the trellises Ts 2 and Tr respectively. 

Theorem 1: For the proposed TCM scheme, the PEP that a 
transmitted path Vs in the trellis is decoded as Vs, under the 
proposed near-ML decoder, is upper bounded by ([Tol l (shown 
at the top of the next page). 

Proof: The PEP that a transmitted path Vs is decoded as 
V s at D is 



Pr{v s — >Vs}= Pr {Ps -+fs\V R )Pr{V R \Vs}: 

Vr£A l 



where Vr is the path to which R decodes. The probability 

Pr {Vr\Vs} is upper bounded by (fTTT) . 
Also, 



Pr [V s — >Vs\P R } 



r l L 

Pr\ min V & (p s ,P R ) > min J" MPs ,Pr)\ (Ps ,Pr) 

(18) 

Let T\ and T<i be two sets defined as, 



Pj?e-At i= i 



> „min y2^(Vs,V R )\ 



^ = 1 ( y di' L ' Y l L ) \J2<t>iCPs,V R )> . min V) <fc fP s , P H ) L 
l V ' 7 fet v R eA L fri J 

Since J 7 ! C T2, from ( TT~8b we have, 

Pr[Ps —>Ps\P R ) 

I L L 

<Pr\Y,<t>i(Ps,V R )> „min V «fc(p s , (V s , Pr) 

(19) 

Let T3 be the set defined as, 

**= U {(y^ L ^; L )\J2^(ps,pR)>J2MPs,pR)}- 



From (19[ , noting that C J-3 and using the union bound, 
we have, 

Pr{Ps — >Vs\P R ) 

^ H p r{^4>i{Ps,P R )>^4>i(Ps,V R )\{Vs,P R )\ 



(20) 



The probability inside the summation in d20l > can be upper 
bounded using Lemma 1. The constant c, the vectors Xi, i € 
{1,2} and the vector y , defined in Lemma 1, are chosen as 
given by (f2Tb-(f24l> (shown at the top of the next page). 



Using Lemma 03 in ( l20l > gives rise to d25l ) (d25l)-(l27l) are 
shown on the next page). Substituting ( fTTT i and d25b in ( TPTb 
gives d26l i. Taking expectation in d26i i with respect to h l sd , 
Kid > ^rd anc ^ ^sr> 1 — * — L, yields ( f27b which is the same 
as ( fl6l i. 

■ 

The criteria to maximize the diversity order of the proposed 
TCM scheme is obtained in the following subsection. 



(17) B. Diversity Criteria 




Path? 



Path? 



Fig. 7. Example illustrating the notion of unmerged length between two 
paths 



Definition 1: The unmerged length between the paths V\ 
and V2 in the trellis, denoted as h(Vi,V2) is the number of 
branches in which the paths V\ and V2 differ. For example, for 
the paths V\ and V2 of length 6 shown in Fig. [7] the unmerged 
length h(Vi,V2) = 5. The unmerged length of the code is the 
minimum value of h(Vi,V2), over all possible pairs of paths 
(Vi,V 2 ) in the trellis. 

Note 3: Since the unmerged length between a pair of paths 
ViiVi is the same in all three trellises Ts ± , Ts 2 and Tr, we 
simply refer to it as the unmerged length in the trellis, without 
mentioning whether the trellis is Ts x , Ts 2 or Tr. 

For two paths Vs and Vs whose unmerged length is equal 
to the unmerged length of the code, the first term in the 
PEP upper bound given by ( [ToT l is of diversity order upper 
bounded by twice the unmerged length of the code, since 
both \v sl {V s ,V s )\ and \Vs 1 (Vs,V S )Ut 1 AVs,V s )\ are upper 
bounded by the unmerged length of the code. Hence, the 
diversity order of the proposed TCM scheme is upper bounded 
by twice the unmerged length of the code. The mappings X$ 1 , 
Xs 2 and Xr should be chosen such that this bound is met with 
equality. 

For two arbitrary paths Vs and Vs in the trellis, let the 
diversity order corresponding to the error event {Vs — > "Ps*j 
be denoted as D(Vs,Vs). 

Corollary 1: For the proposed TCM scheme, D(Vs, Vs) is 
bounded as, 



Pr[Vs —> Vs) < 



n 

ierj B1 (-P S ,P S ) 



V 



1 + 



<?sd (x^ (V S ) -<(P S )) 



n 

ie{i3 2 < p s.T's) u 'J»-< p s.^s)} 



1 + 



3ti (x* 2 (Ps) - x* 2 CP s )) I + |"V<i (x* (Ps) - xj.(P s )) 



+ E E 



n 



1 + 



° s d (< (Ps)~<(Ps)) 



n 

ie{ris 2 (P S .T's) L J'J»-<PH,7 : 'R)} 



1 + 



".d (la 2 Cs) - < 2 CPs)) f + |<T rd (xj. (Vr) - X«(# K )) 



n 

i e{isi< T 's. T 'j?) u ''=i( p s.^H)} 



1 + 



(16) 



c = iE (\ h sr ~ <(^s))f - |h* r «(7>7i) - <(Ps))| 2 ) (21) 

i=l ^ ' 

(22) 

^2 = [^ dl < (Ps), ^ dl < (Ps), -,h L adl x^ (Vs), hl d2 xl 2 (V s ) + h} rd xl d (V R ), h 2 ad2 x 2 S2 (Vs) + h 2 d x 2 d (V R ), h L sd2 x L S2 (V s ) + h L rd x L rd (V R )\ T 

(23) 

V = [Yl,Yl,...,Y d L l ,Yl,Yl,...,Y d L 2 ] T (24) 



Pr 



Pr 



{v s 



{v s 



Pr 



{v s 



V s \Vr} < J2 CX P IE {-\ K<*i « - < (^s)) f - \ \h' sd2 (x' 2 (P S ) - x* 2 CP s )) + /,*„ (xj. (V R ) - xj.(P K ) 

-\ \Kr « (PR) ^ < (Ps)) f + i |C « (P TC ) - x* si (P S )) Q } 

i>s}< E E ex p{E(-5| ft U (^ 1 (Ps)-x« 1 (P s ))| 2 -ik* d2 ^(p^-X^CPs^+^^xt^-x'^R) 

-\ \K T «(Ph) -<(Ps))| 2 - i |>4 «(Ps) - ^ s1 (Ph))| 2 )} 

^}< e e n 



(25) 



(26) 



^ sd (x' ai (Vs) - (x ai (V s )) 



°sd (x\ 2 (Vs) - (xl 2 (Vs))\ + \<y rd (x« (P R ) - x« (P K ) 



1 + 



<Jsr (x* x (P H ) - x* x (Ps)) | + \vsr (xl^Vs) - x* ai (Vr) 



(27) 



Pr (Vs —>Vs)<K II 

iev si (p s ,-p s ) 



\<?sd (x« si CPs) - (xj^Ps))! 



n 

ie{»7s 2 (•P s ,P l s)U^r(7's.^s)} 



(x« 2 (V S ) - (xl 2 (V s ))\ + |<T.d (xl(V s ) - x* r (Vs)) 

(28) 



|r? sl (Ps,^s)l + \Vs 2 {T S ,fs)\ < D(V S ,Vs) 

< min {\ris 1 (Vs,Vs)\ + \Vs 2 (Vs,V s )VJnr{V s ,Vs)l 

2\r)s 1 {'Ps,Vs)\ + \r,s 2 {Vs,Vs)\} , 

(29) 

with equality on both the sides if and only if ij r (Vs, Vs) is a 
subset of r] S2 (Vs,V s ) ■ 

Proof: From ( [Tol l. D(Vs,Vs) is given by, 

D(Vs,Vs) = \Vs 1 (Vs,V S )\+ min { \r, a2 (V s , V s ) U r) r (Vr, Vr)\ 

Vr,Vr 1 

+k S i(^H,^s)U7? Sl (7'«,7's)l}, 

> [»7« 1 (P»,'Ps)[+ min \Vs 2 (V s ,V s )U Vr (V R ,V R )\ 
V R ,Tr 

+ min I^CPh.P^U^^Ps)!. 
Vr.Vr 

(30) 

The quantity |7y S2 (V s , Vs) U f] r (VR,VR)\ attains the min- 
imum value, for the case when Vr = Vr, in which case 
VriVRjpR) is an empty set, or for the case when the set 
Vr{VR,VR) is a subset of f] S2 (V s ,Vs)- In both the cases the 
minimum value attained is \rj S2 (V s , Vs) \- 

The minimum value of \rj ai (Vr, V S )^V S1 {Vr, V s )\ equal 
to zero is attained when Vr — Vs and Vr = Vs- 

Hence from (f30b we get the lower bound in ( [29) . 

The lower bound in (|29T > is met with equality if both the 
minima in (|3Q| > are attained for the same choice of the pair 
(Vr,Vr), which occurs if and only if Vr — Vs, Vr = Vs 
and the the set ^(VrjVr) is a subset of rj S2 (Vs,Vs)- In 
other words, equality in (l30T > occurs if and only if rj r (V s ,V s) 
is a subset of ij S2 (Vs,Vs)- 

The choice Vr ~ Vs and Vr = Vs results in, 

\Vs 2 (Vs,Vs)UVr(V R ,V R )\ 

+ \Vs 1 (VR,?>s)Ur 1 s 1 (V R ,Vs)\ = \Vsi{Vs,fs)U-nr{Vs,fs)\- (31) 

The choice Vr = Vr = Vs results in, 

\Vs 2 (Vs,V S )UVr(V R ,V R )\ 

+ \v*i (Pr, Ps ) U Vai (Vr , V S ) | = | Vs 2 (V s , V S ) | + (Vs , Vs) | . 

(32) 

From (l3TT l and (l32l i we get the upper bound in (1291 . 

Equality in both the lower and upper bounds of (l29b occurs 
if and only if r] r (Vs,Vs) is a subset of rj S2 (Vs,Vs)- ■ 

Definition 2: The effective length of a pair of paths 
(Vs,Vs) in the trellis 7si is defined to be the cardinality 
of the set r) Sl (Vs,Vs)- The effective length of the trellis Ts x 
is defined to be minimum among the effective lengths of all 
possible pairs of paths (Vs, Vs) - In a similar way, the effective 
lengths of the trellises Ts 2 and Tr can be defined. 

Definition 3: The generalized effective length of a pair of 
paths (Vs,Vs) in the trellis pair (Ts 2 ,Tr) is defined to be 
the cardinality of the set {r] S2 (V s ,V s ) ^ -q r (V s ,Vs)\- 



Note 4: If the labellings X S2 and X r are the same, the 
generalized effective length of the pair of paths reduces to the 
effective length of the pair of paths in the trellis Ts 2 , which is 
equal to the effective length of the pair of paths in the trellis 
Tr- 

If X Sl and X S2 are chosen such that the effective lengths of 
the trellises and Ts 2 are equal to the unmerged length of 
the code, then for every pair of paths {(Vs, Vs), |?7 Sl (Vs, Vs)\ 
and \rj S2 (Vs,Vs)\ are greater than the unmerged length of 
the code. In that case, from the lower bound in Corollary 1 
and from the fact that the diversity order cannot exceed twice 
the unmerged length of the code, it follows that the diversity 
order for the proposed scheme is equal to twice the unmerged 
length of the code. Hence, a sufficient condition to obtain 
maximum diversity order is to maximize the effective lengths of 
the trellises Ts x and Ts 2 , independent of the labelling X r . This 
is not surprising, since during Phase 2 the source encodes the 
same bits it encoded and transmitted during Phase 1, maximum 
diversity order is obtained irrespective of whether the relay 
transmits during Phase 2 or not. But since a^d ^ a rd> f rom 
( l28| > it can be seen that the coding gain is greatly reduced 
if the relay does not transmit during Phase 2. It is shown in 
the following subsection that the effective length of Tr also 
should equal the unmerged length of the code to avoid this 
reduction in coding gain. 

Even though the above condition guarantees maximum 
diversity, there is no simple expression for D(Vs,Vs) (Corol- 
lary 1 gives only the upper and lower bounds) and hence it 
is not clear which error events contribute to the minimum 
diversity order. Proposition 1 below gives the conditions under 
which an error event results in the minimum diversity order. 

Proposition 1: Assuming that the sufficient condition to 
obtain full diversity is satisfied, the error event corresponding 
to a pair of paths contributes to the minimum diversity order 
if and only if the effective length of the pair of paths in Ts 1 
and the generalized effective length of the pair of paths in 
(Ts 2 ,Tr), are equal to the unmerged length of the code. 

Proof: Let Vs and Vs be two paths in the trellis. The 
maps Xs 1 and Xs 2 are chosen such that the effective lengths 
of the trellises Ts\ and Ts 2 are equal to the unmerged length 
of the code. 

From the lower bound in Corollary 1, D(Vs,Vs) is greater 
than or equal to twice the unmerged length of the code, since 
\r]s 1 ('Ps,Vs)\ and \ri S2 (Vs,Vs)\ are greater than or equal to 
the unmerged length of the code. Equality can occur only 
when the set r] r (Vs,Vs) is a subset of f] S2 (Vs,Vs), and 
the effective lengths of the pair (Vs,Vs) m both the trellises 
T Sl and T S2 , i.e., \r) sx (Vs,V s )\ and \rj S2 (V s ,V s )\, are_equal 
to the unmerged length of the code. Clearly, ri r (Vs,Vs) is 
not a subset of rj S2 (Vs,Vs), if and only if \rj r (Vs, Vs) U 
Vs 2 (Vs,Vs)\ > \rj S2 (Vs,Vs)\, in which case the generalized 
effective length of the pair of paths (Vs,Vs) in the trellis pair 
(7s 2 , Tr) is greater than unmerged length of the code. Hence, 
the error event corresponding to a pair of paths give rise to the 
minimum diversity order, if and only if the conditions given 
in the statement of the Proposition are satisfied. ■ 



In the rest of the paper, it is assumed that the sufficient 
condition to obtain maximum diversity is satisfied. 

For a pair of paths Vs and Vs resulting in the minimum 
diversity, only the first term in ( [ToT l gives rise to minimum 
diversity order. Neglecting terms of higher diversity order in 
(fl6l l. we get (l28l i (shown on the previous to the previous page), 
where if is a positive constant. 

The criterion to maximize the coding gain is obtained in the 
following subsection. 

C. Coding Gain Criterion 

Let mi(Vs,Vs) denote the product distance of the pair of 
paths (Vs,Vs) in the trellis Ts 17 i.e., 

mi(P s ,P s ) = n K C p s)-x\ 1 {Vs)"\. 

Definition 4: The generalized product distance of the pair 
of paths (Vs,Vs) corresponding to the trellis pair (Ts 2 ,Tr) 
denoted as m,2{Vs,Vs) is defined as, 

m2{v s ,r s ) = EI (-y\4 2 CPs) -< 2 (p s )\ 2 

i£{Vs 2 (V s ,Vs)Ur, r (V s ,V s )} 

+ \4 (P s )-<(^s)| 2 ) , 

(33) 

where 7 = a 2 sd /a 2 d . 

Note 5: If the labelling used for the trellises Tr and Ts 2 are 
the same, the generalized product distance of the pair of paths 
(Vs,Vs) corresponding to the trellis pair (Ts 2 ,Tr), reduces 
within a constant scaling factor to the product distance of the 
pair of paths (Vs,Vs) in the trellis Tr, which is equal to the 
product distance of the pair of paths (Vs>Vs) in the trellis 
T~s 2 - 

Definition 5: Consider the trellis triplet (Tsi , Ts 2 ,Tr). The 
combined product distance of the pair of paths (Vs,Vs), 
denoted by m{Vs,Vs), is defined as the product of the product 
distance of the pair of paths (Vs,Vs) in the trellis Ts 1 and 
the generalised product distance of the pair of paths (Vs,Vs) 
corresponding to the trellis pair (Ts 2 ,Tr), i.e., 

m{V s ,V s ) = [mi(Vs,Ps)m 2 (Ps,Vsj\ . 

Definition 6: The combined product distance of the trellis 
triplet (Ts! , Ts 2 , Tr), which is also the coding gain metric de- 
noted by Q, is defined to be the minimum value of m(Vs, Vs), 
i.e., 

5= min \m(Vs,fisj\ ■ 

(V s ,V s )eZ-p L J 

where Z-p denotes the set of all pairs of paths (Vs,Vs) whose 
effective length in 7sj and the generalized effective length in 
(Ts 2 , Tr) are equal to the unmerged length of the code. 

From ( 128b . it can be seen that the combined product distance 
of the trellis triplet (7s 1 ,7s 2 ,Tr) needs to be maximized, to 
maximize the coding gain. For a pair of paths (Vs, Vs) which 



contribute to the minimum diversity order, if the effective 
length of (Vs,Vs) in the trellis Tr is not equal to the 
unmerged length of the code, then x].(Vs) = x l r {Ps) for some 
i E J]r(Vs,Vs)- In that case m2{Vs,Vs) is greatly reduced 
since 7 <C 1. In order to avoid this, the effective length of the 
trellis Tr should be made equal to the unmerged length of the 
code. 

D. Near Optimality of the Proposed Near-ML decoder 

The following argument proves the high SNR near optimal- 
ity of the proposed near-ML decoder for the TCM scheme. 
Consider the situation where the S-R link is ideal, i.e., the 
relay decodes all the bits correctly. The optimal ML decoder 
at D decides in favour of the path given by, 

/ L . ,2 
V S = arg ^min £ | - h\ dl x\ x (V s ) \ 
L \i= 1 

+ \yI - K d2 < 2 CPs) - K d K (Ps)f) . 

For this case, the PEP that the path Vs is decoded as Vs 
is upper bounded by (l34t (shown at the top of the next page). 
Taking expectation with respect to the fading coefficients, we 
get d35l > (shown at the top of the next page). From (f28b and 
(l35l l. we see that the high SNR bounds on the PEP for the 
proposed near ML decoder with a non-ideal S-R link is same 
as that of the optimal ML decoder with an ideal S-R link, if 
we consider only the error events giving rise to the minimum 
diversity order. The simulation results presented in Section VII 
confirm that the BER vs SNR performance of the proposed 
near ML decoder with a non-ideal S-R link approaches the 
performance of optimal ML decoder with an ideal S-R link, 
at high SNR. The high SNR performance of the optimal ML 
decoder for the proposed TCM scheme, with a non-ideal S-R 
link cannot be better than that of optimal ML decoder for the 
case when the S-R link is ideal. This implies the high SNR 
performance of the proposed near-ML decoder approaches the 
performance of the optimal ML decoder. 

From (l28l . it is clearly seen that even for the uncoded 
transmission scheme, choosing the labelling schemes used at S 
and R properly can potentially yield a significant performance 
improvement. The effect of the choice of labelling on the 
performance for the uncoded transmission scheme is discussed 
in the following section. 

V. THE UNCODED TRANSMISSION SCHEME 

Consider the uncoded transmission scheme in which bits 
are directly mapped onto complex symbols at S and R. A 
collection of log^M bits constitutes a message. Let Ai = 
{1,2,..., M} denote this message set. The uncoded transmis- 
sion scheme has an equivalent one state trellis representa- 
tion, with M edges connecting two successive stages. The 
unmerged length between every pair of distinct paths is one 
and hence no distinction needs to be made between paths and 
edges. The index i in the transmitted complex numbers 
x z S2 (.) and x*(.) can be dropped and they are the same as the 



Pr[Vs 
Pr (Vs 



-Ps} <»p{E(-i| h U (<(^)-(<(^))| 2 -]J|^ 2 ( 



| |^d 2 Vi 1 <•••'., <r. .) /-;.;(.-; .p- - Or.-,) 



(34) 



Vs) <K Yl 



s d (a*! CPs) ~ (^ S1 (P S ))| 

n 



(35) 



labellings on the edges <-t S2 (.) and A" r (.). Each path 

(edge) in the trellis is identified by the message seM which 
gets transmitted. 

Corollary 2: For the uncoded transmission scheme, the PEP 
that the decoder at D decides in favour of message a £ M. 
given that the message transmitted by the source was a £ M. 
is upper bounded as, 

Pr (a — > a) < 



1 + -ksd| 2 |zsi(a) - x si (a)\ 2 



1 + jWsd\ 2 \^s 2 (a) - a; S2 (a)| 2 + j\a rd \ 2 \x r (a) - x r (a)\ 2 



H.O.T. 



where H.O.T denotes the terms of diversity order greater than 
2. 

Proof: From ( fTol l. replacing the paths Vs, Vs, Vr and 
Vr by the corresponding message indices, we get d36*i l (shown 
at the top of the next page). Neglecting terms of diversity order 
greater than 2 gives the result. ■ 
From Corollary 2, it follows that the diversity order of the 
uncoded transmission scheme is two and in order to minimise 
the PEP, we need to maximize the following: 



1 + -Wsd\ 2 \^s 1 (a) - x 31 (a)\ 2 
1 + ^Wsd\ 2 \xs 2 (a) - z S2 (a)| 2 + jWrd\ 2 \x r {a) - x r (a)\ 2 



At high SNR we need to maximize the metric, 

m(a, a) =\x si (a) - x si (a)| 2 

[■y\x S2 (a) - x a2 (a)| 2 + \x r (a) - x r (a)\ 2 ] , 

over all message pairs (a, a). 

By a labelling scheme, we refer to the triplet (X Sl , X S2 , X r ). 
Let C denote a labelling scheme used at S and R. Let us define, 

d(C) = min m(a,a), 

a,a,a^a 

where a, a £ JA. 

Let Cq denote the labelling scheme in which the mapping 
from bits to complex symbols, used by S (during Phase 1 and 



Phase 2) and R (during Phase 2), are the same. Similar to 
d(C), d(Co) can be defined for the labelling scheme Cq. 

Definition 7: The Labelling Gain of the labelling scheme 
C, which is a measure of the performance gain provided by 
C over Cq, is given by, 



d(C) 



d(C ) 



dB. 



L G {C) =101og 10 

It is important to note that the Labelling Gain is calculated 
based on the upper bound on the PEP, taking into consider- 
ation only those pair of messages a and a which contribute 
dominantly to the metric m(a,a). The actual high SNR gain 
provided by the labelling scheme C over the scheme Co need 
not equal Lq(C). 

Throughout, the phrase with our labelling means that S and 
R use the labelling scheme which is described in the following 
subsection and with constant labelling means that S and R use 
the labelling scheme Cq. 

A. A Labelling Scheme for PSK Constellation 

In this subsection, we provide a good labelling scheme for 
2 ( -PSK constellation, where I > 2 is an integer. Let Sfc = 
exp(jfc27r/M), where < k < M - 1 and M = 2 l , denote 
the signal points in the 2 ( -PSK signal set. 

Let C denote the labelling scheme described as follows: The 
maps X Sl , X r and X S2 are given by, 



X S1 (k) 

X r {k) = 

X S2 (k) 



s (k + M/2 mod M) i 



if k is even, 
if k is odd, 



: Sfc, 



where < k < M - 1. 

Theorem 2: For the labelling scheme C, with 2' -PSK signal 
set, the labelling gain, assuming 7 <C 1, is approximately given 
by, 



201og 10 
201og 10 



cot(7r/2')] , for / = 2,3; 
4 cos 2 (tt/2 1 )] , for / > 4. 



(37) 



Proof: Let us define, 

d 1 (k,k') = \X sl (k)- X si (k')\ 

= I expO'fc2?r/Af) - exp(jk' 2n / M)\ 
= |1 - exp(j(k' - k)2n/M)\, 

where < k, k' < M - 1. 



Since di(k, k') depends only on \k— k'\, let us denote it by 
di(n), where < \k — k'\ = n < M . Then, 



di(n) = |1 - cxp(jn2w/M)\ 
= 2| sin(7rn/M)|, 

where < n < M. Similarly, let us define, 



(38) 



d(Cn) ~ min df (n) 

l<n<M 

= 16sin 4 (7r/Af). 



(44) 



Hence, from d43l ) and d44b . by definition of Lg(£), we have, 



d 2 (n) = \X r (k) - X r (k')\ 

|1 — exp(jn2n/M )|, for n even 
jl + cxp(jn27r/Af)|, for n odd 

2| sin(7rra/M)|, for n even 
2| cos(7rn/M)|, for n odd 

Since 7 <C 1, we have 



d(£) rj min |AT sl (fc) - AT S1 (fc')| 2 |A" r (fc) - ^(fc')! 2 
= , niin ^<i?(")<il(w) 

1 < n < M 

= -, J"!?, r ( mil1 (") d 2 ' ^1 (™) d 2 ("•)])• 

Kn<M n even n odd 



From d38l > and d39i > we have, 



min [d 1 (n)d 2 (n)l = min [16 sin (irn/M)] 

l<n<M l<n<M 
n even n even 



16sin 4 (27r/M) 



(39) 



(40) 



(41) 



and similarly, 



L a (£) = 201og 10 [min(cot(7r/M),4cos 2 (7r/M))]. 

It can be verified that cot(7r/M) > 4cos 2 (n/M) for M > 
12, from which we get d37| ). ■ 

It can be seen from Theorem 2 that for I — 2, the labelling 
gain Lg{C) « dB. In other words, for the uncoded transmis- 
sion scheme using 4 PSK, using different labelling schemes 
at S and R does not provide any performance improvement. 
It can be seen that cot(7r/2 z ) for I = 3, and 4cos 2 (7r/2') for 
I > 4 are greater than one, whence the approximate value of 
Lg{C) is greater than dB for I > 3. Since Lg{C) is always 
greater than the approximate value given by Theorem 2, for 
the uncoded transmission scheme using 2 l PSK, using our 
labelling scheme provides advantage over constant labelling, 
for all I > 3. Furthermore, since 4cos 2 (7r/2 z ) is an increasing 
function of I, the labelling gain increases with increasing 
size of the PSK constellation. Simulation results presented in 
Section VII confirm that the uncoded transmission with our 
labelling outperforms the uncoded transmission with constant 
labelling. 



min [di(n)d 2 (n)] 

l<n<M 
rt odd 



min [16sin i! (7rn/M)cos i! (7rn/M)] 

l<n<Af 
n odd 

4sin 2 (27r/M). 



Substituting (04]) and g2]> in (0O]l we have, 



VI. TCM CODE DESIGN GUIDELINES AND 
EXAMPLES 

( 42 ) A. TCM Code Design Guidelines 

Let the generalized product distance m2{Vs 1 Vs) be split 
into two parts as, 



d(C) a min[16sin 4 (27r/M),4sin 2 (27r/Af)]. 



(43) 



m,2('Ps,'Ps) = rn2i(Vs,f's) + m22^Ps,'Ps), 



Also, we have 



where the product distances rn2i(Vs,Vs) and rri22('Ps,'Ps) 
are given by, 



m 21 (V S ,V S )= II \4CPs)-4(JPs)\ , 

m 2 2(V s ,V s )= n l\< 2 {Vs)-< 2 {Vs) ] \ ■ 
Let Qi and Q2 be two metrics defined as, 

Q\ = min mi(?s,^s) 1 

62= min mi('Ps,'Ps)"i2l('Ps,'Ps) , 

where J^p denotes the set of pairs of paths whose effective 
length is equal to the effective length of the trellis Ts-i- 

The code design guidelines are summarized below. Some 
of the design rules for TCM for the point to point AWGN and 
fading channels ( ED, ED), carry over for the relay channel 
as well. 

> Since trellises with parallel transitions limit the effective 
length of the trellis to one, they should be avoided. 

• Signal points from the signal set should occur with equal 
frequency. 

• In a trellis with regularity and symmetry, two branches 
which emerge from the same state form a part of a pair 
of paths which differ by a unmerged length equal to the 
unmerged length of the code. Since the coding gain metric 
Q involves parameters mi(Vs,Vs), m2i(Vs,Vs) and 
m 22('Ps,'Ps), which are of the form of product of the 
branch Euclidean distances of the pair of paths (Vs, Ps)> 
signal points assigned to branches emerging from the 
same state should be from the same Ungerboeck partition 
ED- 

Since 7 <C 1, the values of Q and Q2 are nearly equal and 
instead of maximizing Q, Q2 can be maximized. In the design 
examples presented, the choice of X Sl is made such that the 
effective length of Ts x is maximized and the value of Q\ is 
made large. The choice of X S2 and X r are made such that 
the effective length of Ts 2 is maximized and the value of Q 2 
is made large. In the design examples presented, while the 
diversity order obtained is maximum, it is not claimed that 
the value of Q2 for all the cases is maximum. As was the case 
with TCM for fading channel, maximizing the coding gain is 
a separate problem in itself, as it heavily depends on the trellis 
and the signal set used. 

B. Code Design Examples 

In all the trellis diagrams shown, the labellings on the edges 
are shown to the left of each state. The triple inside [.], when 
read from left to right, denotes the labellings (X Sl , X r , X S2 ) 
corresponding to the edges emerging from top to bottom (the 
labellings X Sl , X r , X S2 are shown in the same trellis diagram 
instead of three different trellis diagrams). In Examples 2, 
3, 4 and 5 considered in this subsection, S transmits two 
information bits to D in the two phases of relaying. 

Example 2: Consider the case where encoding at S and R 
take place using a two state trellis (Fig. |9]l with 8 PSK signal 




Fig. 8. 8-PSK signal set 




Fig. 9. Two State Trellis 

set with signal points labelled as shown in Fig. [8] Let 8q, 
Si, S2 and £3 denote the ordered squared Euclidean distances 
between the points in the 8 PSK signal set, i.e., So = (2 — 
s/2) = 0.586, 5-l = 2, 8 2 = (2 + \/2) = 3.414 and 8 3 = 4. 
Since the unmerged length of the code is equal to one, the 
diversity order cannot exceed two. The choice of X Sl shown 
in Fig.|9]results in the value of the effective length of the trellis 
Ts 1 equal to one and maximizes Q\. The value of Q\ = 83 = 4. 
Choosing X r and X S2 same as X Sl results in the value of the 
effective length of Ts 2 equal to one and maximizes Q2 - For this 
example, the diversity order is 2 and the coding gain metric 
G = 83(83 + 7<5 3 ) = 16.5056, for 7 = 0.0316. 

Example 3: Consider the case where encoding at S and R 
take place using a four state trellis (Fig.flOli with 8 PSK signal 
set. The unmerged length of the code is two and hence the 
diversity order cannot exceed four. From [22|, the choice of 
X Sl shown in Fig. [10] results in a value of the effective length 
of 7si equal to two and Q\ = 8083 = 2.344. 

If X Sl , X S2 and X r are chosen based on the design criteria 
for the fading channel (Jamali et al. labelling ll22l ). the value 
obtained for the metric Gi = (^o^) 2 = 5.49. Instead, if the 
labellings X Sl , X S2 and X r are chosen as shown in Fig. [Tolthe 
value of Q2 — 8^818283 = 16 and the effective length of Ts 2 
equal to two. For this example, the diversity order is 4 and 
the coding gain metric Q = 5o^3(^i^2 + 7^3) = 16.1735, 
for 7 = 0.0316. 

Example 4: Consider the case where encoding at S and R 
take place using an eight state trellis (Fig. fTTT i with 8 PSK 
signal set. For this trellis, the diversity order cannot exceed 



[(s 2 ,s 4 ,s 2 ) (s 6 ,s 2 ,s 6 ) (s 8 



[( s 7' s y s 7 ) < s 3' s r s 3 ) ( s 5' s 5' s 5 ) ( s v s 3' s i)l 11 



[(s 4 ,s 2 ,s 4 )(s 8 ,s 4 ,s 8 )(s s ,s 8 ,s 6 )(s 2 ,s 6 ,s 2 )] 10 i 




take place using a sixteen state trellis shown in Fig. [T2] The 
unmerged length of the code is three and hence the diversity 

00 order cannot exceed six. From ll22ll . the choice of X Sl shown 
in Fig. [12] (which is the same as Ungerboeck's labelling |20|) 
results in a value of the effective length of 7si equal to three 
and Qi = 5q5i53 = 4.68. Labellings X r and X S2 chosen 

01 to be the same as X Sl result in Q2 — (^o^i^) 2 = 21.96 
and the effective length of 7s 2 equal to three. For this 
example, the diversity order is 6 and the coding gain metric 

11 Q= (£ <M3) 2 (1 + 7) = 22.66, for 7 = 0.0316. 



10 



Fig. 10. Four State Trellis 

four since the unmerged length of the code is two. From l23ll . 
the choice of X Sl shown in Fig. QTJ (which is the same as 
Ungerboeck's labelling [20]) results in a value of the effective 
length equal to two and Q\ — = 8. Choosing X r and 
X S2 same as X Sl results in Q2 = (^i^) 2 = 64 and ensures 
that the effective length of in Ts 2 is equal to two. For this 
example, the diversity order is 4 and the coding gain metric 
Q = (5i<5 3 ) 2 (l + 7) = 66.0224, for 7 = 0.0316. Increasing 
the number of states from 4 to 8, while it provides an increase 
in the coding gain, does not provide diversity advantage. 



KWtHW^VAM 8 /'*^ 000 
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[(WrHWsKWsHV^l 101 



KWaXW^W^W^ 100 



Fig. 11. 8 State Trellis 

Example 5: Consider the case where encoding at S and R 




0000 
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Fig. 12. 16 State Trellis 

In the following example we consider the case where 3 bits 
of information get transmitted from S to D in the two phases 
of relaying. 

Example 6: Consider the case where encoding at S and R 
take place using an eight state trellis (Fig. [T4l with 16 PSK 
signal set whose points are labelled as shown in Fig. [13] For 
this trellis the diversity order cannot exceed four, since the 
unmerged length of the code is two. The choice of X Sl shown 
in Fig. [14] results in a value of Q\ = 0.0892 and ensures that 
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Fig. 14. 8 State Trellis 




Fig. 13. 16 PSK Signal Set 



the effective length of 7si is two. The labellings X r and X S2 
shown in Fig. [14] result in Q2 = 0.1177 and ensures that the 
effective length of 7s 2 is two. For this example, the diversity 
order is 4 and the coding gain metric Q = 0.1295, for 7 = 0.1. 

VII. SIMULATION RESULTS 

The Eg vs BER plots for the different schemes achieving a 
spectral efficiency of 1 bpcu are shown in Fig.[l5j for <j 2 sd = 
dB, crf r = 15 dB and of d = 15 dB. The diversity order and 
the value of E$ required at a BER of 10~ 4 are summarized 
in Table U As observed in the previous section, the uncoded 
transmission scheme provides a diversity order 2. We observe 
that using a 2 state trellis provides a diversity order 2 as 
expected. At a BER of 10~ 4 , the 2 state 8 PSK TCM scheme 
provides 4.5 dB gain over the uncoded transmission scheme 
using 4 PSK. When the number of states is increased to 4, 




D red Tx. - BPSK 
0< Direct Tx. - 2 State TCM 
•» D reel Tx. - 4 State TCM 
■tV-Relay Channel - Uncoded Tx. Scheme 
A Relay Channel - 2 State TCM 
V Relay Channel - 4 State TCM - Our Labelling 
-X-Relay Channel -8 State TCM 
Q +Relay Channel- 16 State TCM 

'•a — Rela * Channel " 4 S1 * TCM-Jamali et al Labelling 



-10 -5 5 10 15 20 25 30 35 
E s in dB 



Fig. 15. E$ vs BER for the uncoded transmission scheme using 4 PSK 

7 sd 



and TCM Schemes for a\ A = dB, a 2 sr = 10 dB and cr^ = 10 dB 



we see that the diversity order increases to 4 as predicted and 
at a BER of 10~ 4 a large gain of 9.5 dB is obtained over 
the uncoded transmission scheme using 4 PSK. Also, it can 
be seen from Fig. Q3J it can be seen that the 4 state 8 PSK 
scheme with our labelling provides a gain of 0.6 dB over the 
case when S and R use Jamali et al. labelling. Increasing the 
number of states to 8, from Fig. [15] the diversity order stays 



TABLE II 

Comparison of uncoded 8 PSK transmission scheme and 16 PSK TCM Schemes 



Scheme 


Diversity Order 


E s in dB for BER = 1CT 4 


Uncoded Transmission Scheme - 8 PSK - With Constant Labelling 


2 


20 dB 


Uncoded Transmission Scheme - 8 PSK - With Our Labelling 


2 


18 dB 


Relay Channel - 8 State TCM 


4 


12.5 dB 



ORelay Channel -8 Slate TCM 
■U Uncoded Transmission Scheme - 8 PSK With Our Labelling 
+ Uncoded Transmission Scheme - 8 PSK With Constant Labellinc 
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Fig. 16. Eg vs BER for the uncoded transmission scheme using 8 PSK 
and TCM Schemes for a 2 sd = dB, a 2 sr = 10 dB and a 2 d = 10 dB 



at 4 but a coding gain of 1.5 dB is obtained over the 4 state 
8 PSK TCM scheme. With the 16 state 8 PSK TCM scheme, 
a diversity order of 6 is obtained and a gain of 1.5 dB is 
obtained over the 8 state 8 PSK TCM scheme. In essence, at 
a BER of 10~ 4 , a large gain of 12.5 dB is obtained using the 
16 state 8 PSK TCM scheme over the uncoded transmission 
scheme using 4 PSK. 

Fig. [TBI shows the plots comparing the uncoded transmission 
scheme using 8 PSK with our labelling and with constant 
labelling, and the 8 state 16 PSK TCM scheme, for a 2 d = 
dB, a 2 r = 10 dB and af d = 10 dB. At a BER of 10~ 4 the 
uncoded transmission scheme using 8 PSK with our labelling 
and 8 state 16 PSK TCM provide a gain of 2 dB and 7.5 
dB respectively over the 8 PSK uncoded transmission scheme 
with constant labelling. The diversity order and the value of 
Eg required at a BER of 10~ 4 are summarized in Table [II] 

Fig. [17] shows the E s vs BER plot for the 4 state 8 PSK 
TCM scheme, for the cases where the S-R link is ideal and 
non-ideal, for a 2 sd = dB, a 2 r = 10 dB and a 2 rd = 10 dB. As 
observed in Section IV E, the high SNR performance of the 
proposed near-ML decoder with non-ideal S-R link approaches 
the performance of the optimal ML decoder with an ideal S-R 
link. 



■0-4 State TCM - Ideal S-R link 
■EJ4 Slate TCM -Non-Ideal S-R link 



E s ,nd 



Fig. 17. Eg vs BER comparison for the proposed TCM scheme with ideal 
and non-ideal S-R links for a 2 d = dB, a 2 r = 10 dB and a 2 d = 10 dB 



VIII. DISCUSSION 

A TCM scheme for the half duplex fading relay channel was 
proposed. A near-ML decoder whose high SNR performance 
approaches the performance of the optimal ML decoder was 
obtained. Based on the expression for PEP bounds, code 
design criteria to maximize the diversity order and coding gain 
were formulated. Maximizing the diversity order can be done 
by a proper choice of the labellings used at the source during 
Phase 1 and Phase 2. A unified procedure to find the labelling 
scheme which maximizes the coding gain for all signal sets 
and trellises is not known and will be an interesting topic for 
further research. 
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